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learning environment, learner attributes, and technological characteristics. 
The overall mean effect size was 0.147, a small positive effect in favor of 
distance education. Effect sizes were more positive for interactive distance 
education programs that combine an individualized approach with traditional 
classroom instruction. Programs including instruction delivered via 
telecommunications, enhancement of classroom learning, short duration, and 
small groups yielded larger effect sizes than programs using 

videoconferencing, primary instruction via distance, long duration, and large 
groups. Studies of distance education for all academic content areas except 
foreign language resulted in positive effect sizes. This synthesis supports 
the use of interactive distance education to complement, enhance and expand 
education options because distance education can be expected to result in 
achievement at least comparable to traditional instruction in most academic 
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The Effectiveness of Interactive Distance Education Technologies in K-12 Learning: A 

Meta- Analysis 

Catherine S. Cavanaugh 
University of South Florida, Tampa 

This paper summarizes a quantitative synthesis of studies of the effectiveness of 
interactive distance education using videoconferencing and telecommunications 
for K-12 academic achievement. Effect sizes for 19 experimental and quasi- 
experimental studies including 929 student participants were analyzed across 
sample characteristics, study methods, learning environment, learner attributes, 
and technological characteristics. The overall mean effect size was 0.147, a small 
positive effect in favor of distance education. Effect sizes were more positive for 
interactive distance education programs that combine an individualized approach 
with traditional classroom instruction. Programs including instruction delivered 
via telecommunications, enhancement of classroom learning, short duration, and 
small groups yielded larger effect sizes than programs using videoconferencing, 
primary instruction via distance, long duration, and large groups. Studies of 
distance education for all academic content areas except foreign language 
resulted in positive effect sizes. This synthesis supports the use of interactive 
distance education to complement, enhance and expand education options 
because distance education can be expected to result in achievement at least 
comparable to traditional instruction in most academic circumstances. 

Between 1989 and 1996, the number of instructional computers in schools increased over 
200% and by 1997, the number of schools with Internet access had reached 70% (Software 
Publishers Association, 1998). In today’s education climate, use of current technology becomes 
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increasingly critical in schools for several reasons. Schools that are able to demonstrate 
innovative educational strategies using technology are at a distinct advantage in attracting and 
keeping top students, and in earning further funding through grants, endowments, and programs. 
Of course, having technology does not ensure effective use of the tools, and therefore may not 
translate into education benefits. Schools are pressured by their communities to show that the 
investment is paying dividends in educational achievement. Educational administrators and 
decision-makers are challenged with providing increased educational opportunities without 
increased budgets. Many educators are responding to this challenge by developing distance 
learning projects. 

As of 1995, 60% of all personal computers in K-12 schools were networked to another 
computer. Half of all school districts were using distance learning in the “business of education”: 
academic modules and credit courses (CCA Consulting, 1996), driven by demand for time and 
place independence and by economic issues. Eighty two percent of the states reported that 
equality of educational experience in all schools was the principal need addressed by distance 
education projects that provide courses to homebound and remotely located learners (Quinn & 
Williams, 1987). Distance learning applies physical technology and education processes to serve 
the needs of students when they are removed from the source of instruction and resources by 
either time or distance. “As with most instructional tools, the purpose of distance learning is to 
help schools meet the instructional needs of their students and to enable students to access 
information more effectively and apply what they learn in school to the world in which they live” 
(ESA, 1995). Distance learning uses a group of systems to bring teaching and learning together 
by transmitting information or expertise from one place to another for learner benefit. Formally, 
distance education is characterized by physical separation of learners from the information, an 




2 



4 



organized instructional program, use of technological media, and two-way communication 
(Heinich, Molenda, & Russell, 1993). 

The benefits of distance education as outlined by Kerka (1996) include flexibility to meet 
specific needs, providing equity of educational opportunity to students in varying localities, low- 
cost alternatives, new learning experiences, and expanded resources. Organizational learning 
forms infrastructures to enhance and extend the capabilities of the individual to collaborate and 
to utilize specialized knowledge of others (Molnar, 1997). All forms of education are enhanced 
by the increased access to information and communication afforded through distance education. 
Disadvantages of distance education include sound and video that may be less than broadcast 
quality, reliance on learner initiative to work in a situation with less supervision than a 
classroom, the need for technical skills to work with the delivery technology, and the possibility 
of social isolation (Kerka, 1996). 

Distance acquisition of knowledge is often an expensive and time-consuming process to 
institute and maintain. It is important to know whether it actually improves student performance. 
It is also critically important to know which distance education delivery methods and techniques 
are more effective, so students get maximum benefit from society’s investment in distance 
learning technology. Faced with so many new, rapidly changing options, schools need data in 
order to make quality decisions regarding distance education. 

A review of distance education literature reveals well-developed theory supported by 
extensive experimental study. However, distance education research has not been subjected to 
repeated review and synthesis, especially in regards to K-12 education. Holmberg’s theory of 
distance teaching (1985) states that distance teaching will support student motivation and 
promote learning pleasure and effectiveness if learners are engaged in discussions and decisions, 
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and the program provides for real and simulated communication to and from the learners. As in 
any classroom, interaction is the core of teaching. Distance education is believed to work very 
well, and produce results as effective as traditional classroom instruction (Kearsley, 1996). 
However, the distance education currently in practice has the potential to provide more effective 
learning with updated pedagogy, more experience, and greater understanding and knowledge of 
methods. Numerous studies have evaluated student achievement in specific distance learning 
programs in higher education. According to Abrami and Buras (1996), learning at a distance is 
seldom superior to traditional instruction, particularly for promoting higher level achievement 
and complex skills. These weaknesses may stem from social and intellectual isolation. Improved 
distance education practices have the potential to enhance educational outcomes, especially when 
the amount and kind of learner interaction is increased using technology-supported collaborative 
learning. 

Although distance learning is well documented with adults, fewer studies of effectiveness 
exist that center on the primary and secondary levels. At a point when all states offered distance 
education in schools, very few had conducted formal evaluations (Quinn & Williams, 1987). 
While it is possible that our knowledge of distance education technology in higher education is 
relevant with precollege learners, there is no empirical evidence for this assertion. Additionally, 
previous research has spread across grade levels, content areas, and technologies. Distance 
education program developers have paid less attention to the appropriate blend of media, content, 
learner, and gain; instead defining effectiveness in terms of the number of students served or 
student satisfaction (Eiserman & Williams, 1987). 
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Purpose of the Meta-Analysis 



In light of the lack of previous meta-analyses on the issue, the purpose of the current study 
was to arrive at a comprehensive assessment of the effectiveness of interactive distance 
education for K-12 learners. This meta-analysis had two main goals. The first goal was to 
determine the effects on K-12 student achievement of interactive distance education use of 
videoconferencing or on-line telecommunications. Traditional reviews of distance education 
literature conducted in the 1980’s indicate that learners achieve as well in distance education 
programs as they do in traditional classroom settings (Moore, 1989). Other studies described by 
Moore and Thompson (1990) indicate that the instructional format itself has little effect on 
student achievement as long as the delivery technology is appropriate for the content, and timely 
teacher-to-student feedback is included. This research involved technology that is now outdated, 
and was not conducted using quantitative synthesis methods. Good distance teaching practices 
have been found to be fundamentally identical to good traditional teaching practices, with quality 
factors being universal across environments and populations (Wilkes & Burnham, 1991). This 
means not only that various studies of distance learning may be synthesized reliably, but that the 
results of such a synthesis ought to generalize to most comparable education situations. Meta- 
analyses have been performed recently to show trends correlating use of technology in general 
with improved achievement (Kulik, 1993; Ryan, 1994), but not focused on distance education. 

The second main goal of this study was to identify the features (duration of use, frequency of 
use, instructional design, delivery system, ability of students, level of students, content area) of 
the most effective interactive distance education systems. The Software Publishers Association 
reported evidence that educational technology’s positive effects depend on subject area, student 
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population, and level of access to technology (1998). The present quantitative synthesis of 
studies is an effort to increase understanding of the effectiveness of distance learning on K-12 
achievement by revealing the features and combinations of characteristics that make distance 
education most effective. 



Method 

Data Collection 

Studies involving student academic achievement as a result of distance education at the 
K-12 level were assembled. Quasi-experimental studies were included with experimental studies 
because true experiments are done in artificial situations, while quasi-experiments usually occur 
in more realistic conditions. Quasi-experiments, although having lower internal validity, produce 
more realistic results and increased external validity (Carlberg et al, 1984). The setting for the 
studies is K-12 learning environments, with preference for studies using random sampling. Both 
published and unpublished studies are included to avoid publishing bias. 

Studies were gathered from computer databases including ERIC, Dissertation Abstracts, 
requests from national and state distance education projects and the World Wide Web. Searches 
were systematically conducted in journals of education organizations ( American Journal of 
Distance Education, Journal of Educational Computing Research, Journal of Research on 
Computing in Education ) from the 1980’s and 1990’s because of the recent development of the 
distance education delivery systems being examined here. Searches employed the keywords 
“distance education”, “distance learning”, “teleconferencing”, "videoconferencing", “satellite”, 
“telecommunications”, and “email”. The technology directors for each state were contacted by 
email with requests for data. These initial searches unveiled references for hundreds of 
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documents. Each document was located in ERIC files, on the Web, in journals, by interlibrary 
loan, or by ordering dissertations. 

After examination, studies were included in the meta-analysis when they met all inclusion 
criteria: a focus on the use of interactive distance education technology for education at K-12 
levels, either videoconferencing or on-line telecommunications; publication between 1980 and 
1998; experimental or quasi-experimental design providing quantitative outcomes from which 
effect size can be estimated; freedom from obvious flaws. The bibliographies of studies that did 
not meet all inclusion criteria provided references for locating other studies. When studies lacked 
data for inclusion in this analysis, a direct request was made to the authors or sponsoring 
organization to send additional information. A total of 59 such requests were sent via post or 
email. A preliminary group of 59 studies was selected from all available studies for review to 
determine their appropriateness for inclusion in the meta-analysis. Of the 59 studies reviewed, 40 
were eliminated for reasons including insufficient data on student achievement, failure to focus 
exclusively on K-12 learners, focus on attitude rather than achievement. 

Coding and Interrater Agreement 

Studies included in the meta-analysis were reviewed and coded independently by two 
doctoral students. The general categories of coding were publication characteristics, 
technological characteristics, ecological characteristics, methodological characteristics, and 
analytical results. Interrater agreement for all studies averaged 85%, and exceeded 80% for all 
categories of coding. In cases where agreement fell below 80%, the study was examined and the 
coding process was clarified. Percent agreement was calculated for each study by dividing the 
number of agreements by the total number of features coded, then multiplying by 100. 
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Classification of Variables 

The dependent variable in this synthesis was student achievement in the academic content 
area measured by instruments appropriate to the individual study. The measure of student 
performance in the studies was indicated on achievement measures given at the end of the 
distance education period. Such measures included standardized tests and teacher/researcher 
designed instruments. Traditional measures may not be effective in assessing the effectiveness of 
distance education systems that strengthen higher order skills. Complex skills such as problem 
solving, motivation, writing, collaboration, and awareness of community are some of the benefits 
of using distance education (Pisapia & Perlman, 1992). Growth in these areas can only be 
inferred with the use of common achievement measures. The student performance measures used 
in the studies analyzed here are accepted as the best estimates of the effect of interactive distance 
education technology on student achievement currently available. Dependent measures were 
coded into one of eight content areas, one of three types of achievement measure, and one of four 
testing sequences. The content areas were general, language arts, mathematics, science, social 
studies, computer science, and foreign language. The types of achievement measure were 
standardized, teacher-developed, researcher-developed, and other (publisher-developed or final 
course grade). The testing sequences were pretest-posttest or posttest only. 

The treatment variables related to interactive distance education were categorized by 
duration, frequency, instructional design, and delivery system. Student characteristics including 
ability level and grade in school were also coded. Comparisons were made between distance 
education programs in which students used the technology for less than a half-year of instruction 
and those in which the technology was used for at least a half year. Programs exposing students 
to distance education activities almost daily were contrasted with programs in which students 
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participated in distance education on a weekly basis. Within instructional design, studies were 
coded as either using distance education as the primary instructional delivery system, or as using 
distance education systems to supplement regular classroom instruction. The distance education 
delivery system was email, World Wide Web, or two-way audio-video conferencing. Student 
groups were coded on the grade level designations primary, intermediate, middle, or high school, 
and were classified as functioning at regular ability levels or advanced. 

Effect Size Calculation 

The achievement data reported in each study contributed to the calculation of effect size for 
the meta-analysis. Cohen's effect size is a standardized estimate of the difference in achievement 
between students learning with interactive distance education and students learning with 
traditional methods. Effect sizes were computed as the difference between the control and 
experimental posttest mean scores divided by the average standard deviation. For studies in 
which a group of students was evaluated using multiple achievement measures, effect sizes were 
found for each measure, and then averaged. The average effect size was used in the overall 
effect size estimate of the meta-analysis. For studies in which more than one independent group 
of students was evaluated, an independent effect size were found for each group and used in the 
meta-analysis separately. While different groups tested in one study were dependent with respect 
to study features and characteristics, the samples were independent. A correction factor based on 
Hedges, Shymansky, and Woodworth was applied to each biased effect size to remove bias by 
accounting for sample size (1989). The independent effect sizes were combined by weighted 
averaging to yield an overall effect size estimate. 
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Heterogeneity of effect sizes was estimated to determine the appropriateness of combining 
them into an average. A formal analysis of heterogeneity (ANOVA) was completed for the effect 
sizes with effect size as the dependent variable, and the study features as factors. The p-value for 
the ANOVA was also set at 0.01. This procedure enabled conclusions to be drawn regarding the 
significance of levels of variables. For example, the ANOVA may reveal that a certain level or 
type of learner achieves at significantly higher levels using a specific distance education system. 
In addition to the overall student performance, effect sizes were reported for each variable. Effect 
size comparisons were made with respect to the following variables: duration, frequency, 
instructional design, delivery system, ability level, grade level, and content area. Effect sizes 
were also compared for the study characteristics date of study, source of study, hardware 
components, achievement measure, and sample size. 

Results 

The 19 independent effect sizes calculated in this synthesis compared students learning 
with interactive distance education technology with students learning with traditional classroom 
instruction only. The weighted mean effect size across all studies was 0.147 (SD = .69). The 
mean sample size for the studies was 48.9 (SD = 13.2). The mean grade level of the students was 
8.3, with a standard deviation of 0.59. The average study measured student achievement after 
14.7 weeks of distance education (SD = 2.6). Studies using two-way audio-videoconferencing as 
the distance education system comprised 68% of the total, followed by 26% of studies that used 
email, and one study using the World Wide Web. The most common academic content area for 
the studies was science, with approximately 32% of the studies. All other content areas made up 
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less than 16% each. All studies took place in school classroom environments. All studies except 
two involved regular level learners, and the remaining two studied advanced learners. 

The average year of publication for the studies was 1993 (SD = .71), with dates ranging from 
1986 to 1997. The most frequent source of studies was dissertations (68%), followed by journals 
{Journal of Educational Computing Research, American Journal of Distance Education, 
Interactive Learning Environments), and the World Wide Web. Approximately 84% of authors 
were affiliated with universities. 

In terms of the experimental design, two-thirds of the studies used a quasi-experimental 
design, and one-third were experimental. Only 10% of the studies made use of random 
assignment to groups, while 53% used convenience groups such as school classes. The 
remaining studies grouped students based on other criteria such as volunteering or parent 
selection. A full 74% of studies employed a pretest-posttest design, and 36% used posttest only. 
The most common form of measurement instrument in the studies was researcher-developed 
tests (32%), followed by teacher-developed and standardized (26% each), and publisher- 
developed tests (16%). 



Table 1. 

Mean unbiased effect sizes by study variable 



Variable 


Level 


# of Studies 


% of Total 


Average Effect 
Size 


Delivery System 


Telecommunications 


6 


32 


0.488 




Videoconferencing 


13 


68 


-0.011 


Duration 


Over 1 5 weeks 


8 


42 


0.094 




Less than 15 weeks 


11 


58 


0.186 


Frequency 


Daily 


13 


68 


-0.010 




Weekly 


6 


32 


0.489 


Instructional 

Design 


Primary 


13 


68 


-0.011 




Supplementary 


6 


32 


0.489 



11 
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Grade Level 


9-12 


13 


68 


-0.011 




3-8 


6 


32 


0.489 


Content Area 


Science 


6 


32 


0.069 




Social Studies 


3 


16 


0.117 




Language Arts 


3 


16 


0.432 




Foreign Language 


3 


16 


-0.801 




Mathematics 


2 


11 


0.758 




Other 


2 


11 


0.811 


Year 


Pre-1993 


8 


42 


0.225 




Post- 1993 


11 


58 


0.090 


Achievement 

Measure 


Standardized 


5 


26 


0.279 




Non-Standardized 


14 


74 


0.099 


Test Sequence 


Pretest-Posttest 


14 


74 


0.295 




Posttest Only 


5 


26 


-0.268 


Sample Size 


Fewer than 26 


8 


42 


0.454 




Over 32 


11 


58 


-0.076 


Study Source 


Published 


6 


32 


0.590 




Unpublished 


13 


68 


-0.057 



Distance Learning Environments 

For analysis, the effect sizes were classified into 12 categories related to features of the 
studies. Table 1 displays the mean unbiased effect sizes for these categories. Effect sizes above 
Cohen's threshold for "marginal" magnitude (below 0.45) occurred for studies of 
telecommunication, weekly use of distance education, use of distance education to supplement 
classroom instruction, mathematics and other subject areas (computer science and general 
elementary), and intermediate grade levels. Effect sizes above Cohen's 0.80 cutoff of "large" 
effect size were found only for the other subjects, and for foreign language in a negative 
direction. Seven studies in this synthesis had heterogeneity values above 5.3, indicating that they 
were "strikingly deviant". To further investigate the significant heterogeneity, analysis of 
variance of effect sizes was used. An ANOVA was computed for levels of variables shown in 
Table 1. Each ANOVA resulted in no significant difference between levels of the variables at the 
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alpha level of 0.01. Even after the removal of the foreign language studies as outliers, the 
ANOVAs resulted in non-significant findings. The alpha values for the modified ANOVAs are 
presented in Table 2. 



Table 2. 

Modified ANOVA Results 



Study Feature or Characteristic 


ANOVA Alpha Value 


Duration of Use of Distance Education 


0.965 


Frequency of Use of Distance Education 


0.407 


Instructional Design 


0.406 


Technology Delivery System 


0.406 


Student Grade Level 


0.406 


Content Area 


0.437 


Research Design 


0.261 


Study Date 


0.776 


Study Source 


0.171 


Achievement Measure 


0.387 


Sample Size 


0.129 



The heterogeneity in the studies is unexplained. The optimal method for combining 
effect size estimates with unexplained heterogeneity is the random effects model (Hedges et al, 
1989). Random effects procedures treat between-study variations in effect sizes as random. The 
procedures begin with the estimation of between-study variance, used to determine the difference 
between the observed variance among effect sizes and the within-study sampling variance. The 




13 15 



sample variance for the 19 studies in the meta-analysis is 0.481. The between-study variance was 
found to be 0.413. This variance was used in calculating confidence limits and in combining the 
effect size estimates in the random effects model. The effect size formula was weighted with 
both the within-study sampling error variance (standard error) and the variance. 

The overall effect size for the 19 studies is 0.147, considered a small positive result. The 
standard error of the effects weighted average is 0.159. This result is a small positive effect size. 
At an alpha level of 0.05, the 95% confidence limits were constructed by multiplying the square 
root of the variance, 0.413, by 1.96, then adding and subtracting from the effect size, d + (Cooper 
& Hedges, 1994). In this case, the confidence limits were -1.113 and 1.407. Because the interval 
encompasses zero, the null hypothesis that the population effect size is zero cannot be rejected. 
Interactive distance learning techniques cannot be considered to be more or less effective than 
traditional instruction. 

Because of the near-significant result of the ANOVA for subject area and the cluster of 
outliers, a second overall effect size was estimated without the foreign language studies. The 
random effects effect size estimate for the meta-analysis was 0.344, discounting foreign language 
studies. The effect size improves after accounting for these discrepant studies in post hoc 
analysis, although the confidence interval of -0.686 to 1.374 still encompasses zero. The overall 
standard deviation for the random effects effect size estimate was found. The result for the 19 
studies is a standard deviation for effect size of 0.480. 

Discussion of Findings 

The meta-analysis presented here examined a sample of studies that met established inclusion 
criteria, including data from 929 learners. The analysis encompassed a broad view of K-12 




14 is 



interactive distance education by focusing on the range of subject areas, grade levels, and 
applications of distance education in use today. The questions of the overall effect on K-12 
academic achievement of interactive distance education, and of the features of effective distance 
education systems were addressed through the use of random effects effect size estimation and 
analysis of variance. The study design permitted review of a wide sample of research done on 
academic achievement of students learning with distance education. 

The overall effect size for interactive distance education on K-12 learning was the small 
positive 0.147, accounting for sampling error and variance. The 95% confidence interval for the 
random effects weighted effect size was -1.113 to 1.407. In comparing levels of the variables 
delivery system, grade level, ability level, content area, instructional design, duration, and 
frequency of use, and the study features of date, source of study, achievement measure, testing 
sequence, and sample size, no significant differences were found. In a post hoc analysis after 
eliminating the outlier foreign language studies, an effect size of 0.344 was calculated, with a 
confidence interval of -0.686 to 1.374. 

As a result of the statistical processes applied in the meta-analysis, the overall 
effectiveness of interactive distance education has been estimated as it is used for K-12 learning. 
Interactive distance education was found to have an average effect size of 0.147, translating to a 
gain of 0.147 standard deviations on average for a student at the 50 th percentile learning with 
distance education over traditional methods. No significant differences were detected between 
grade levels, subject areas, ability levels, distance education technology, duration and frequency 
of use of distance education, or instructional design in relation to learning. However, the three 
foreign language studies made a close approach to significance with strong negative effect sizes. 
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Consequently, distance education can be expected to result in achievement at least 
comparable to traditional instruction in most academic circumstances. Educators planning 
implementations of distance education programs should expect no difference in academic 
performance as a result of the use of distance education. More importantly, when implemented 
with the same care as effective face-to-face instruction, distance education programs can be used 
to complement, enhance and expand education options for students, at least at intermediate, 
middle, and upper grades levels. 

Implications 

In a time when standards-based education has taken on a greater real-world, situated 
focus, it has become more important for educators to provide students with authentic connections 
to a learning environment beyond the school boundaries. Interactive distance education is a 
vehicle for extending the reach of student influence into the community, as well as a means of 
including the family and community in a learning conversation. In light of the findings of this 
meta-analysis, supplementing traditional instruction with distance education can enable more 
reality-based learning, with possible achievement gains. The benefits of such educational 
enhancement in terms of attitude and interpersonal workplace skills are unknown at the present 
time. 

Under pressure to present learners with greater opportunities at lower cost, educational 
leaders may find that offering courses to secondary learners via interactive distance education 
effectively enlarges the course catalog and students' worldview at the same time. On a cautionary 
note, distance education for K-12 is much newer and less proven than traditional instruction. The 
fact that only 19 studies were found to be suitable for this analysis is a testimony to that fact. 
Therefore, students and teachers have fewer quality materials and less experience on which to 
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draw when using distance education. It can be argued that as the use of interactive distance 
education grows and expertise develops, academic gains can be expected to increase. 

In the area of foreign language instruction, great potential exists theoretically for linking 
students with native speakers and writers, but the results of the studies reviewed here indicate 
that distance education in foreign language should be studied closely. The three foreign language 
studies in this meta-analysis reported that students learning with distance education systems 
performed demonstrably lower than students learning in traditional classrooms. While foreign 
language options are needed in high schools, distance education courses for foreign language 
instruction should be evaluated very carefully. 
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